Discrete complex correlation device for obtaining subpixel accuracy

ABSTRACT

This invention is directed to an image processing arrangement used to estimate image displacement relative to a reference frame. It comprises a discrete complex correlator, an associated interpolator, and a displacement estimator. The unique nature of the system is its ability to estimate shifts of a fraction of a pixel for sparcely sampled data. This is accomplished by extracting the complex gradients from the gray scale data and in turn correlating the gradients via a discrete complex correlator. The resulting cross correlation function is then interpolated to yield an accurate estimate of the shift. The invention is particularly adapted for use in the autofocus portion of a synthetic aperture radar imaging system.

STATEMENT OF GOVERNMENT INTEREST

The invention described herein may be manufactured and used by or forthe Government for governmental purposes without the payment of anyroyalty thereon.

BACKGROUND OF THE INVENTION

The present invention relates generally to synthetic aperture radarimaging systems and more specifically to an autofocus image processingsystem for estimating image displacement relative to a reference frame.

A large class of problems involving image processing are a result of theneed for an accurate registration cability. This task has beenalleviated to some degree by the prior art techniques given in thefollowing patents:

U.S. Pat. No. 4,330,833 issued to Pratt et al on 18 May 1982;

U.S. Pat. No. 4,244,029 issued to Hogan et al on 6 Jan 1981;

U.S. Pat. No. 3,955,046 issued to Ingham et al on 4 May 1976;

U.S. Pat. No. 3,943,277 issued to Everly et al on 9 Mar 1976;

U.S. Pat. No. 4,162,775 issued to Voles on 31 Jul 1979;

U.S. Pat. No. 4,368,456 issued to Forse et al on 11 Jan. 1983;

Pratt et al disclose a method and apparatus for digital image processingwhich operates on dots or "pixels" with an operator matrix havingdimensions smaller than a conventional operator. It may be used in therestoration improvement of photographs or other images taken bysatellites or astronauts in outer space and then transmitted to earth.Hogan et al disclose a digital video correlator in which a referenceimage and a live image are digitized and compared against each other ina shifting network to determine the correlation between the two images.In Ingham et al phase shifts are detected and used to follow a target.Correlation type trackers are also disclosed in the Everly and Volespatents. Forse et al teach an image correlator in which a referencerepresentation is updated by a control processor when the correlation ofit with a current representation reaches a peak.

In view of the foregoing discussion it is apparent that in the realm ofsynthetic aperture radar imaging systems there exists a need fordevelopment in the area of accurate imaging, particularly if the amountof available data is space. The present invention is directed towardssatisfying that need.

SUMMARY OF THE INVENTION

The present invention provides a correlation system with subpixelaccuracy for sparcely sampled data using a correlator, interpolator anddisplacement estimator. The complex gradient correlator is used tocorrelate the complex gradient data and obtain a pronounced responsefrom the data from the range cells of the radar. Then the interpolatorwill interpolate the resulting cross-correlation. The displacementestimator receives the interpolation result to yield an accurateestimate of shifts a fraction of a pixel for sparcely sampled data.

It is an object of the invention to provide a new and improved FeatureReferenced Error Correction (FREC) autofocusing system but itsusefulness is by no means limited to that alone. Any displacementestimate for a gray scale (detected) data relative to a reference framecan be carried out in the same manner as described in this disclosure.When the data is substantially oversampled (as it may be for a directphotograph of a scene) the increased complexity, due to the need togenerate complex gradients, cannot be justified and as such other moreconventional correlation schemes may suffice. Thus for marginallysampled image the discrete complex correlation scheme offers asubstantial subpixel accuracy improvement at the expense of somewhatmore demanding processing.

It is a principle object of this invention to provide a new and improvedcorrelation system with subpixel accuracy for sparcely sampled data.

These together with other objects, features and advantages of theinvention will become more readily apparent form the following detaileddescription when taken in conjunction with the accompanying drawingwherein like elements are given like reference numerals throughout.

DESCRIPTION OF THE DRAWINGS

FIG. 1 is an illustration of the use of an autofocus in syntheticaperture radar processing;

FIG. 2 is a functional block diagram of one embodiment of the presentinvention;

FIG. 3 is an illustration of the Sobel Window;

FIG. 4 is a graph of the discrete complex correlator response in twodimensions;

FIG. 5 is an illustration of the discrete complex correlator response inthree dimensions;

FIG. 6A is the response of the complex correlator to real subaperturedata for zero shift correlation;

FIG. 6B is the response of the complex correlator to real subaperturedata for non zero shift correlation; and

FIG. 7 is a set of charts depicting the interpolation process and itseffects on a signal as processed by the three steps of interpolation.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

This invention is directed to an image processing arrangement used toestimate image displacement relative to a reference interpolator, and adisplacement estimator. The unique nature of the system is its abilityto estimate shifts of a fraction of a pixel for sparcely sampled data.This is accomplished by extracting the gradients via a discrete complexcorrelator. The resulting cross-correlation function is theninterpolated to yield an accurate estimate of the shift. The inventionis particularly adapted for use in the autofocus portion of a syntheticaperture radar imaging system.

Autofocus is a processing technique that extracts information from thepartially processed data to yield an estimate of the error phase presentin the data. This, in turn, is used to remove the phase errors from thedata prior to its final processing.

A number of autofocus techniques have been developed that successfullyestimate the error phase, but with various degrees of processingcomplexity. Techniques that utilize the fully processed image (incomplex form), and that require multiple passes to achieve the finalfocus, have been successful but cumbersome and therefore not practicalin a real-time environment. Other variations of the multipass techniquesusing iterative search have been successful yet suffer from the samenon-real-time constraint.

A different approach is the Feature Reference Error Correction (FREC)technique, which is based on the requirement of a single pass andintegration with existing Synthetic Aperture Radar (SAR) processing.

FIG. 1 is an illustration of the use of a single pass, feed forwardautofocus in use for typical synthetic aperture radar processing. Radardata is input from range processing into the First Stage Fast FourierTransform (FFT) 101. The output of FFT 101 is the subaperture data whichneeds to be correlated in a specific manner to yield the error phasewhich can be removed.

The correlation is done after the data enters the bulk memory 102 by theautofocus 103. The autofocus 103 uses the partially processed dataresiding in the bulk memory 102 to extract the error phase with theresult that the phase errors are removed 104 from the data and sent tothe second stage FFT 105 prior to final processing.

The autofocus 102 extracts the error phase by correlating thesubaperture data in a specific manner to yield the shifts relative to areference subframe. These shifts are then reconstructed in a way whichregenerates the complete error phase across the full aperture. Startingwith data (for a single subframe and a reference subframe) the processinvolves the following steps:

a generation of the complex gradient of the detected data;

b. line per line complex correlation with an ensemble average over allrange cells;

c. interpolating the data block to obtain the shift estimate; and

d. estimation of the displacement between the two subframes.

The steps followed by the autofocus in extracting the error phase may beaccomplished completely by software on a high speed data processor byfollowing the procedure described below, or the steps may beaccomplished by the combination of software and the hardware equivalentsdepicted in FIG. 2.

FIG. 2 is a functional block diagram of one embodiment of the presentinvention. In FIG. 2, the functions of the autofocus 103 of FIG. 1 areaccomplished by the following:

a data processor 200 performs the functions of complex gradientgeneration 201 and complex correlation 202 (using the process describedbelow);

an interpolator 300 consists of a Fast Fourier Transform (FFT) 301, aZero Filling Device 302, and an Inverse FFT 303; and

the Displacement Estimator 400 consists of a multiplier 401 and either aLeast Square Fit 402 or an integrator. These functional hardware blocksperform the process described below which may also be accomplishedentirely in software by a high speed data processor.

After the autofocus 103 receives the output of the first stage azimuthFFT 101, the complex data is linear detected, noise clipped to yield thepredominant signal (gray scale) and its average intensity is estimated.At this point each subframe is transformed (on a line per line basis)into a complex gradient subframe. This is carried out via a Sobel Windowas given in FIG. 3.

The Sobel window of FIG. 3 is characterized by the function F_(i) asdefined below in Table 1.

                  TABLE 1                                                         ______________________________________                                        x = A.sub.2 + A.sub.4 - (A.sub.0 + A.sub.6) + 2(A.sub.3 - A.sub.7)            y = A.sub.0 + A.sub.2 - (A.sub.6 + A.sub.4) + 2(A.sub.1 - A.sub.6)             ##STR1##                                                                     φ = tan.sup.-1 (y/x)                                                      F.sub.i = A.sub.i e.sup.jφ i = x.sub.i + jy.sub.i                         ______________________________________                                    

The average power of each subframe (computed in conjunction with thegain distribution across the full aperture) is then used to selected athreshold which in turn is used to reject bad correlation lines. Eachline which passes the power threshold is correlated (for 5 to 7 shiftsabout zero shift) and a running (ensemble) average is used to collapseall of the correlation data over all of the range cells utilized. It isthis function which must then be processed further in order to estimateaccurately the associated displacement.

Once the intensity gradient is generated a complex set of numbers areobtained for each subframe. Conventional correlation techniques appliedto the magnitude of the gradient cannot achieve superior performance tothat of conventional intensity correlators. However, when a complexcorrelator is used to correlate the complex gradient data the responseis more pronounced and devoid of ambiguities. The complex correlator isgiven as: ##EQU1##

The algorithm computes the cross correlation (more precisely a matchindex) which is the summation of the gradient vector alignments betweenscene pairs. Here A is the complex gradient for frame A while B is thecomplex gradient of the reference frame B. Note that R_(AB) isnormalized by the total power and furthermore that when A=B (a match)the resultant due to the match summation of the numerator yields a realpositive number. Thus for a perfect match the correlation is positivewhile the range for R_(AB) is between -1 to +1. This algorithm has thecharacteristic of a coherent process in that images that are misalignedproduce very low (essentially zero) correlation value. Only whenalignment is close does the index have non-zero values. The correlationfor correct alignment is the result of coherent summation for intensitygradient pairs which produces a spike like response which peaks at thecorrect match position. FIG. 4 indicates intensity gradient vectorcorrelation behavior for a discrete intensity pedestal which producesthe array of intensity gradients. It is evident that the correlationfunction (i.e., autocorrelation) is spikey and very rapidly settles tozero.

When the technique is applied to two dimensional scenes the resultingfunction retains its essential characteristics of unique peak and rapiddecorrelation away from the peak. A three dimensional plot of theresulting response is given in FIG. 5 where the sharp peak and thebipolar value of the response function is evident.

It can therefore be concluded that the complex correlator possesses theessential ingredients needed for the FREC processing. The only remainingcrucial issue is how to achieve subpixel accuracy for marginally sampleddata. This is the subject of the following section.

The complex gradient correlation process described above, yields (forrealistic data) a very narrow and well defined correlation function(whose positive peak is the only region of interest). This is shown inFIG. 6 for a zero shift correlation (in this case the autocorrelation)and a non-zero shift correlation. It is evident that the correlationfunction is marginally sampled and as such yields a rather crudeestimate of the displacement which is desired to within 1/100 of apixel.

The interpolation procedure is shown in FIG. 6 where 5 or 7 points ofthe correlation functions are considered to be the data. Thisinterpolation procedure is also summarized by the block diagrams of FIG.2 and consists of: doing a FFT 301, adding trailing zeros 302 (asdepicted in FIG. 6), then performing an Inverse FFT 303. In FIG. 6, theFFT of the new data block is carried out, zeros are added to itsmidpoint resulting in a total of 128 pts (for this example of KOSF=8i.e. 16×8=128).

By adding trailing zeros to the data, it is converted to a convenientbinary number. The inverse FFT of this modified spectrum is then carriedout to yield the interpolated data block. Selecting the maximum point ofthis interpolated data block as well as the neighboring 5 points oneither side of it, gives a good description of the peak region. At thispoint a second order LSE fit is carried out on the eleven new datapoints from which one can easily estimate the local peak (for ax²+bx+c=0, x_(p) =-b/2a). The true peak is then related to the originaldata by accounting for the oversampling factor as well as the necessaryindex changes. This interpolation procedure results in an accurate shiftestimate.

The shift estimate from the interpolator is next processed by theDisplacement Estimator 400 of FIG. 2. First the shift estimate isconverted into a phase rate to obtained a displacement history. One wayto accomplish this is simply to multiply the shift estimate by aconstant as seen in 401 of FIG. 2. The result in turn can be integratedto yield the desired error phase estimate. An alternative to integrationis the least square fit 402 which performs a least square estimation toobtain the estimate of the phase error since typically some 32subapertures are used, a good quality phase estimate is possible.

With the accurate phase error obtained by the autofocus 103, using theprocedure described above, the data from the synthetic aperture radarnext has this error subtracted from it as shown by 104 of FIG. 1. Theresult is the removal of phase errors from the data prior to its finalprocessing with the elimination of shifts of a fraction of a pixel forsparcely sampled data.

While the invention has been described in its presently preferredembodiment it is understood that the words which have been used arewords of description rather than words of limitation and that changeswithin the purview of the appended claims may be made without departingfrom the scope and spirit of the invention in its broader aspects.

What is claimed is:
 1. An autofocus device in combination with asynthetic aperture radar system to extract error phase from detecteddata, said autofocus device comprising:a complex gradient generatorreceiving said detected data from said synthetic aperture radar systemand generating a complex gradient from said detected data; a complexcorrelator receiving said complex gradient from said complex gradientgenerator and outputting a line per line complex correlation; aninterpolator receiving said complex correlation from said complexcorrelator and performing an interpolation to obtain a shift estimate;and a displacement estimator receiving said shift estimate from saidinterpolator and converting said shift estimate into said error phase.2. An autofocus device as defined in claim 1 wherein said complexgradient generator is a data processor which generates said complexgradient by performing a Sobel Window process on said detected data. 3.An autofocus device as defined in claim 2 wherein said complexcorrelator is a data processor which performs said line per line complexcorrelation by processing said complex gradient with the followingalgorithm: ##EQU2## where A is the complex gradient for reference frameA; and B is the complex gradient for reference frame B.
 4. A process ofcorrelating detected data from radar range processing to extract anerror phase estimate comprising the steps of:generating a complexgradient of the detected data; complex correlating said complex gradientoverall range cells after said generating step and producing a complexcorrelation; interpolating said complex correlation and producing ashift estimate after said complex correlating step; and estimatingdisplacement of said shift estimate to produce said error phaseestimate.
 5. A process of correlating detected data as defined in claim4 wherein said generating step comprises processing said detected datawith a Sobel Window to produce said complex gradient; andsaid complexcorrelating step comprises producing a complex correlation by processingsaid complex gradient with the following algorithm: ##EQU3## where A isthe complex gradient for reference frame A; and B is the complexgradient for reference frame B.
 6. A process of correlating detectingdata as defined in claim 5 wherein said interpolating stepcomprises:performing a Fast Transform on said complex correlation andproducing a Fast Fourier Transform output signal; adding zeros to saidFast Fourier Transform output signal at its midpoint and producing aconvenient binary number; and performing an Inverse Fast FourierTransform on said binary number to produce a shift estimate.
 7. Aprocess of correlating detected data as defined in claim 6 wherein saidestimating displacement step comprises:multiplying said shift estimateby a constant to produce a phase rate; and performing a least squareestimate on said phase rate to produce said error phase estimate.
 8. Aprocess of correlating detected data as defined in claim 6 wherein saidestimating displacement step comprises:multiplying said shift estimateby a constant to produce a phase rate; and integrating said phase rateto produce said error phase estimate.